Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: FDR Latest pulling at the edge (S3 upload/download) #1994

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

RohinBhargava
Copy link
Member

@RohinBhargava RohinBhargava commented Jan 12, 2025

Short description of the changes made

  • This change introduces pulling FDR Latest specs directly from S3, using already existing S3 asset buckets. The reason we can do this is because we no longer have shape transformations for FDR docs shapes, and instead have one shape that is registered and read.

  • This PR chooses to use FDR to upload S3 specs, however, this can easily be moved into the CLI (before this, there should be a discussion of security concerns of distributing S3 keys to clients/this is not an issue in Next since the api keys are used in the backend part of next). For spec pulling, this is done directly at the site that needs the data (some FDR apis or edge functions, because we need to index algolia records).

*The idea is that this should improve performance, since we see that FDR is a current bottleneck, and that there is one less hop to deliver the data to the end user.

THIS NEEDS TO BE THOROUGHLY TESTED

  • Coupling this with the OpenApi Parser v2 rollout is beneficial, because we can properly flight/test this new approach.

What was the motivation & context behind this PR?

  • Performance issues
  • Future vision to parse data at the edge
  • Integration with dynamic snippets

How has this PR been tested?

  • To be tested in FDR dev.

Copy link

vercel bot commented Jan 12, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
rbac.ferndocs.com ❌ Failed (Inspect) Jan 12, 2025 11:16pm
1 Skipped Deployment
Name Status Preview Updated (UTC)
fern-shell ⬜️ Ignored (Inspect) Jan 12, 2025 11:16pm

@RohinBhargava RohinBhargava had a problem deploying to Preview - app.buildwithfern.com January 12, 2025 23:16 — with GitHub Actions Error
@RohinBhargava RohinBhargava had a problem deploying to Preview - app-dev.buildwithfern.com January 12, 2025 23:16 — with GitHub Actions Error
Copy link

Copy link

Copy link

github-actions bot commented Jan 12, 2025

📦 Next.js Bundle Analysis for fern-platform-monorepo

This analysis was generated by the Next.js Bundle Analysis action. 🤖

This PR introduced no changes to the JavaScript bundle! 🙌

Copy link

github-actions bot commented Jan 12, 2025

PR Preview

Copy link

github-actions bot commented Jan 12, 2025

Playwright test results

passed  96 passed
flaky  3 flaky
skipped  2 skipped

Details

stats  101 tests across 9 suites
duration  1 minute, 37 seconds
commit  faa4a4d

Flaky tests

chromium › forward-proxy/express.spec.ts › home page 404
chromium › skew-protection/index.test.ts › should contain ?dpl= or x-deployment-id header on all scripts and prefetch requests
chromium › smoke/favicon.spec.ts › Check if favicon exists and URL does not return 404 for developers.gappify.com

Skipped tests

chromium › posthog.spec.ts › Posthog loads successfully
chromium › websockets/websocket.spec.ts › websocket test

Copy link

Copy link

Copy link

Copy link

@RohinBhargava RohinBhargava marked this pull request as ready for review January 13, 2025 22:58
Copy link

Copy link
Member

@dsinghvi dsinghvi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a large PR, but this makes sense to me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants